The Cross-Entropy Method

This report is a summary of the theory underlying the Cross-Entropy (CE) method, as discussed in the tutorial by de Boer, Kroese, Mannor and Rubinstein [1]. For a more thorough discussion of the method and its applications, please refer to the original tutorial and the references cited in the tutorial. The CE method, pioneered by Rubinstein in 1997 as an adaptive algorithm for estimating probabilities of rare events, has been broadened as a generic and efficient tool for solving a myriad of NP-hard problems. Beyond its original purpose, the CE method has been employed in deterministic and stochastic combinatorial optimization problems (COPs) and continuous multi-extremal optimization problems. This report is organized as follows. In Section 2, we discuss the fundamental theory of the CE method and specialize the method to rare-event simulation (RES) and COPs. In Section 3, we consider more sophisticated versions of the CE method, and briefly discuss convergence issues. We conclude the report in Section 4.